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ABSTRACT 



The invention enables users to virtually attach information 
to situations in the real world, and also enables users to 
quickly and easily find out desired information. An IR sensor 
receives an IR signal transmitted from an IR beacon, and 
supplies the received signal to a sub-notebook PC. A CCD 
video camera takes in a visual ID from an object, and 
supplies the inputted visual ID to the sub-notebook PC. A 
user inputs, through a microphone, a voice to be attached to 
situations in the real world. The sub-notebook PC transmits 
position data, object data and voice data, which have been 
supplied to it, to a server through a communication unit. The 
transmitted data is received by the server via a wireless 
LAN. The server stores the received voice data in a database 
in correspondence to the position data and the object data. 
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INFORMATION PROCESSING APPARATUS AND 
METHOD, INFORMATION PROCESSING 
SYSTEM, AND PROVIDING MEDIUM 

[0001] This is a continuation application of a U.S. patent 
application Ser. No. 09/413,432, filed Oct. 6, 1999, which is 
incorporated herein by reference. 

BACKGROUND OF THE INVENTION 

[0002] The present invention relates to an information 
processing apparatus and method, an information processing 
system, and a providing medium. More particularly, the 
present invention relates to an information processing appa- 
ratus and method, an information processing system, and a 
providing medium, which are adapted to attach various 
virtual information to real world environments or situations 
(such as a position and an object), and to present those 
various attached information. 

[0003] Recently, systems and interfaces called Augmented 
Reality (abbreviated as AR hereinafter) have been proposed. 
"Augmented Reality" is a generic term for systems and 
interfaces based on the concept of providing information, 
which corresponds to situations in the real world such as a 
position and an object, to users and offering a more com- 
fortable interface for the real world to the users. 

[0004] FIG. 30 shows the concept of AR. As shown in 
FIG. 30, a user terminal, e.g., a computer that the user can 
wear on himself or herself (referred to as a wearable 
computer 122 hereinafter), is always held in an ON-state to 
observe real world environments 121 and to present appro- 
priate information to the user as required. 

[0005] A sensor 123 includes, e.g., an IR (infrared) sensor, 
a CCD (Charge Coupled Device) video camera and a 
microphone to take in information (position information, ID 
information, voice information, etc.) of the real world envi- 
ronments 121. The information taken in by the sensor 123 is 
supplied to a server 125 via a network. The server 125 reads 
data, which corresponds to the supplied information of the 
real world, from an internal database 125a in which data 
corresponding to the information of the real world is stored 
beforehand, and renders the read data to be displayed on an 
HUD (Head Up Display) 124 via a network. With such a 
system, the user can acquire the data corresponding to the 
situations in the real world environments 121. 

[0006] One example of such an AR system is reported by 
Steven Feiner, Blair Maclntyre, and Doree Seligmann, 
"Knowledge-based augmented reality," Communication of 
the ACM, Vol. 36, No. 7, pp. 52-62, August 1993. This AR 
system is a maintenance system for a laser beam printer. 
Assist information produced based on position information, 
which is obtained by a ultrasonic sensor, is indicated on a 
transmission type wearable display mounted on the user's 
head. 

[0007] Apart from the above-described AR system, there 
are known systems allowing users to exchange various 
information about things in the real world or various infor- 
mation in the virtual world via a network or the like. For 
example, information exchange service systems called an 
on-line bulletin board system (BBS) and chat over the 
Internet have been already utilized. With one of personal 
computer (PC) communication services, a bulletin board 
(forum) or the like subdivided depending upon favorable 



items of users is provided so that the users can exchange 
information via the bulletin board. This form of service often 
plays a predominant role in the user community. The BBS is 
usually utilized in such a manner that the user retrieves an 
interesting bulletin board through a search service based on 
the name of the bulletin board, etc., and then browses 
information written on the retrieved bulletin board or writes 
new information on it. 

[0008] However, the conventional AR system described 
above is problematic in that the assist information displayed 
on the HUD or the like is limited to the information 
previously registered in a database stored in a computer, and 
such information (annotation) as popping into the user's 
head during maintenance work cannot be newly added to the 
database. Specifically, the above-described AR system is 
only able to read information corresponding to situations in 
the real world from among the information previously stored 
in the database, and to present the read information to users. 
In other words, the conventional AR system has been merely 
a context- aware browser. 

[0009] When users receive the information exchange ser- 
vices such as the BBS and chat over the Internet or PC 
communication, the users face such a problem that they have 
to retrieve, e.g., interesting bulletin boards through a search 
service based on the name of each bulletin board, etc. 
However, a large number of bulletin boards, etc. exist on the 
Internet, and it is not always easy to quickly find out a 
desired one of those bulletin boards, etc. on which desired 
information is written. 

SUMMARY OF THE INVENTION 

[0010] In view of the state of art set forth above, an object 
of the present invention is to provide an information pro- 
cessing apparatus and method, an information processing 
system, and a providing medium, with which information 
(such as image information, text information and voice 
information) corresponding to situations (such as a position 
and an object) in the real world can be presented to users, 
while users can attach, to situations in the real world, 
information corresponding to the situations in the real world, 
and with which users can quickly and easily find out desired 
information. 

[0011] The above object of the present invention is 
achieved by an information processing apparatus for com- 
municating with another information processing apparatus, 
wherein the apparatus includes a specific information input 
unit for taking in specific information to specify situations in 
the real world; an attached information input unit for taking 
in attached information to be attached to the situations in the 
real world specified by the specific information that is taken 
in by the specific information input unit; a transmitting unit 
for transmitting the attached information and the specific 
information; a receiving unit for receiving the attached 
information transmitted from the another information pro- 
cessing apparatus corresponding to the transmitted specific 
information; and a presenting unit for presenting, as infor- 
mation related to the specified situations in the real world, 
the attached information having been received. 

[0012] Also, the above object of the present invention is 
achieved by an information processing method implemented 
in an information processing apparatus for communicating 
with another information processing apparatus, wherein the 
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method includes a specific information input step of taking 
in specific information to specify situations in the real world; 
an attached information taking- in step of taking in attached 
information to be attached to the situations in the real world 
specified by the specific information; a transmitting step of 
transmitting the attached information and the specific infor- 
mation; a receiving step of receiving the attached informa- 
tion transmitted from the another information processing 
apparatus corresponding to the specific information trans- 
mitted in the transmitting step; and a presenting step of 
presenting, as information related to the specified situations 
in the real world, the attached information having been 
received. 

[0013] Further, the above object of the present invention is 
achieved by an information processing apparatus for com- 
municating with another information processing apparatus, 
wherein the apparatus includes a receiving unit for receiving 
specific information to specify situations in the real world 
and attached information attached to the situations in the real 
world specified by the specific information, the specific 
information and the attached information being both trans- 
mitted from the another information processing apparatus; a 
storage unit for storing the received specific information and 
attached information; a reading unit for reading the attached 
information corresponding to the specific information from 
the storage unit; and a transmitting unit for transmitting the 
attached information having been read. 

[0014] Still further, the above object of the present inven- 
tion is achieved by an information processing method imple- 
mented in an information processing apparatus for commu- 
nicating with another information processing apparatus, 
wherein the method includes a receiving step of receiving 
specific information to specify situations in the real world 
and attached information attached to the situations in the real 
world specified by the specific information, the specific 
information and the attached information being both trans- 
mitted from the another information processing apparatus; a 
storing step of storing the received specific information and 
attached information; a reading step of reading the attached 
information corresponding to the specific information stored 
in the storing step; and a transmitting step of transmitting the 
attached information having been read, 

[0015] Still further, the above object of the present inven- 
tion is achieved by an information processing system includ- 
ing at least a first information processing apparatus includ- 
ing a specific information taking-in unit for input specific 
information to specify situations in the real world, an 
attached information input unit for taking in attached infor- 
mation to be attached to the situations in the real world 
specified by the specific information, a transmitting unit for 
transmitting the attached information and the specific infor- 
mation, a receiving unit for receiving the attached informa- 
tion transmitted from a second information processing appa- 
ratus corresponding to the transmitted specific information, 
and a presenting unit for presenting, as information related 
to the specified situations in the real world, the attached 
information having been received; and a second information 
processing apparatus including a receiving unit for receiving 
the specific information and the attached information both 
transmitted from the first information processing apparatus, 
a storage unit for storing the received specific information 
and attached information, a reading unit for reading the 
attached information corresponding to the specific informa- 



tion from the storage unit, and a transmitting unit for 
transmitting the attached information read by the reading 
unit; the first information processing apparatus and the 
second information processing apparatus being connected 
via a network. 

[0016] Still further, the above object of the present inven- 
tion is achieved by a providing medium for providing a 
computer-readable program to execute a processes including 
the steps of a specific information input step of taking in 
specific information to specify situations in the real world; 
an attached information input step of taking in attached 
information to be attached to the situations in the real world 
specified by the specific information; a transmitting step of 
transmitting the attached information and the specific infor- 
mation; a receiving step of receiving the attached informa- 
tion transmitted from the another information processing 
apparatus corresponding to the transmitted specific informa- 
tion; and a presenting step of presenting, as information 
related to the specified situations in the real world, the 
attached information having been received* 

[0017] Still further, .the above object of the present inven- 
tion is achieved by a providing medium for providing a 
computer-readable program to execute a process including 
the steps of a receiving step of receiving specific information 
to specify situations in the real world and attached informa- 
tion attached to the situations in the real world specified by 
the specific information, the specific information and the 
attached information being both transmitted from the 
another information processing apparatus; a storing step of 
storing the received specific information and attached infor- 
mation; a reading step of reading the attached information 
corresponding to the specific information stored in the 
storing step; and a transmitting step of transmitting the 
attached information having been read. 

[0018] In the present invention, the specific information 
includes at least one of image information of the real world, 
position information of the real world, and object informa- 
tion of the real world. Also, the situation in the real world is 
an object in the real world, the specific information is an ID 
attached to the object in the real world, and the attached 
information transmitted from the transmitting unit is infor- 
mation that is related to the object in the real world and can 
be browsed and exchanged. Further, the situation in the real 
world is a position in the real world, the specific information 
is position information of the real world, and the attached 
information transmitted from the transmitting unit is infor- 
mation that is related to the position in the real world and can 
be browsed and exchanged. 

BRIEF DESCRIPTION OF THE FIGURES 

[0019] FIG. 1 is a block diagram showing a schematic 
entire configuration of an information processing system of 
the present invention. 

[0020] FIG. 2 is an illustration for explaining the concept 
of Augment-able Reality according to a first embodiment of 
the present invention. 

[0021] FIG. 3 is a block diagram showing a configuration, 
of one example of an Augment-able Reality system accord- 
ing to the first embodiment of the present invention. 

[0022] FIGS. 4A and 4B are representations for explain- 
ing a visual ID. 
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[0023] FIG. 5 is a block circuit diagram showing one 
example of configuration of a sub-notebook PC in FIG. 2. 

[0024] FIG. 6 is a representation showing one example of 
a screen image of an HUD. 

[0025] FIGS. 7A to 7C are representations for explaining 
the operation of moving an icon between a personal infor- 
mation tray and a context-aware area. 

[0026] FIG. 8 is a representation for explaining a time 
machine mode. 

[0027] FIG. 9 is a flowchart for explaining the process of 
producing information to be attached to situations in the real 
world and registering the information in a database in the 
Augment-able Reality system. 

[0028] FIG. 10 is a flowchart for explaining the process of 
reading and displaying the information, which has been 
attached to situations in the real world, from the database. 

[0029] FIG. 11 is a representation showing one example 
of a screen image of Java applet in a computer (41) in FIG. 
3. 

[0030] FIG. 12 is a representation showing one example 
of E-mail transmitted from a computer (42) in FIG. 3. 

[0031] FIG. 13 is a schematic view showing one example 
of conceptual configuration of a system according to a 
second embodiment of the present invention. 

[0032] FIG. 14 is a representation showing one example 
of an initial screen image on a client PC in a BBS search 
using an ID. 

[0033] FIG. 15 is a representation showing one example 
of a screen image of a bulletin board retrieved in accordance 
with the ID. 

[0034] FIG. 16 is a representation showing one example 
of a screen image when an ID is newly registered. 

[0035] FIG. 17 is a flowchart showing a process flow 
when retrieval and browsing of and writing on a bulletin 
board corresponding to an article ID are performed, or a 
bulletin board is newly registered on the client PC side in the 
system according to the second embodiment. 

[0036] FIG. 18 is a flowchart showing a process flow 
when the bulletin board corresponding to the article ID or a 
request for new registration of a bulletin board is transmitted 
to the client PC from the server side in the system according 
to the second embodiment. 

[0037] FIG. 19 is a flowchart showing a process flow on 
the server side when a number of bulletin boards corre- 
sponding to the same ID are found by a search. 

[0038] FIG. 20 is a flowchart showing a process flow on 
the client PC side when a list of the names, abstracts, etc. of 
the number of bulletin boards are transmitted from the server 
side. 

[0039] FIG. 21 is a schematic view showing one example 
of conceptual configuration of a system according to a third 
embodiment of the present invention. 

[0040] FIG. 22 is a representation showing one example 
of a display screen image of a bulletin board retrieved using 
the specific name of some restaurant as a search condition at 
the current position. 



[0041] FIG. 23 is a representation showing one example 
of a display screen image of a bulletin board retrieved from 
bulletin boards at respective positions near the current 
position in match with additional search conditions. 

[0042] FIG. 24 is a flowchart showing a process flow for 
new registration of a bulletin board related to the current 
position. 

[0043] FIG. 25 is a flowchart showing a process flow for, 
e.g., retrieval and browsing of and writing on a bulletin 
board corresponding to the current position. 

[0044] FIG. 26 is a flowchart showing a process flow on 
the server side when new registration of a bulletin board 
corresponding to the current position is requested from the 
client PC. 

[0045] FIG. 27 is a flowchart showing a process flow for, 
e.g., retrieval and browsing of and writing on a bulletin 
board in relation to the current position. 

[0046] FIG. 28 is a flowchart showing a similarity deter- 
mining algorithm in a method of using, as a "position" of 
convenience, a pair of base station ID and electric field 
intensity when position recognition is carried out using the 
PHS. 

[0047] FIG. 29 is a block circuit diagram showing one 
example of concrete configuration of principal parts of client 
PCs (304, 404) applicable to the second and third embodi- 
ments. 

[0048] FIG. 30 is an illustration for explaining the concept 
of conventional Augmented Reality. 

DESCRIPTION OF THE INVENTION 

[0049] Preferred embodiments of the present invention 
will be described with reference to the drawings. 

[0050] FIG. 1 shows an entire configuration of one work- 
ing mode of the present invention in which an information 
processing apparatus and method, an information processing 
system, and a providing medium are embodied according to 
the present invention. 

[0051] Referring to FIG. 1, a first information processing 
apparatus (which is primarily used by users and is referred 
to as a user terminal 200 hereinafter) is shown, which 
includes a real-world-environment information recognizing 
unit 201 which serves as specific information input unit 
means for taking in specific information to specify situations 
(context information such as information regarding an object 
image, an object ID, a position and a voice) in the real world 
environments 1; a real -world- environment information ana- 
lyzing unit 202 for analyzing the information taken in by the 
real-world-environment information recognizing unit 201; 
an attached information input unit 203 for taking in attached 
information to be attached to situations (such as an object 
and a position) in the real world; a communication unit 207 
for communicating information between the user terminal 
200 and a second information processing apparatus, for 
example, (which is primarily installed on the service pro- 
viding side and is referred to as a server 210 hereinafter); 
and a control unit 204 for receiving desired information 
from the server 210 based on the information analyzed by 
the real-world-environment information analyzing unit 202, 
and for producing a video signal, a voice signal, etc., based 
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on the desired information. The user terminal 200 further 
includes a display unit 206 for displaying the video signal 
from the control unit 204 on a screen, and a voice output unit 
205 for outputting the voice signal from the control unit 204. 
Additionally, the functions of the user terminal 200 in this 
working mode may be implemented in the form of hardware 
or software. 

[0052] Furthermore, in this working mode of the present 
invention, the server 210 includes an information accumu- 
lating unit 213 for storing the specific information to specify 
situations in the real world and the attached information 
attached to the situations in the real world which are 
specified by the specific information, both types of the 
information being transmitted from the user terminal 200; a 
communication unit 211 for receiving the specific informa- 
tion and the attached information transmitted from the user 
terminal 200, and for transmitting, to the user terminal 200, 
the attached information read from the information accu- 
mulating unit 213 corresponding to the specific information; 
and a control unit 212 for controlling processes of storing 
and reading information in and from the information accu- 
mulating unit 213 and processes of transmitting and receiv- 
ing data to and from the communication unit 211. Addition- 
ally, the functions of the server 210 in this working mode 
may be implemented in the form of hardware or software, 

[0053] In this working mode of the present invention, the 
user terminal 200 and the server 210 are connected to each 
other via a network, for example, to construct an information 
processing system (network system). 

[0054] As a more concrete form, a first embodiment of the 
network shown in FIG. 1 will be described below with 
reference to FIGS. 2 to 12. 

[0055] An information processing apparatus (e.g., a user 
terminal) according to the first embodiment of the present 
invention is an information processing apparatus for com- 
municating with another information processing apparatus 
(e.g., a server), and mainly includes a CCD video camera 21, 
shown in FIG. 3, as one example of an image input unit for 
taking in an image of the real world; an IR sensor 22, shown 
in FIG. 3, as one example of a position detecting unit for 
detecting a position in the real world; an image processing 
unit 61, shown in FIG. 5, as one example of an object 
identifying unit for identifying an object in the real world; a 
microphone 24, shown in FIG. 3, as one example of an 
attached information input unit for taking in attached infor- 
mation, which is to be attached to the real world, corre- 
sponding to the position information outputted from the 
position detecting unit or the object information outputted 
from the object identifying unit; a step S7, shown in FIG. 9, 
as one example of transmitting unit for transmitting the 
attached information taken in by the attached information 
input unit, the position information and the object informa- 
tion; a step S29, shown in FIG. 10, as one example of a 
receiving unit for receiving, from the another information 
processing apparatus, the attached information transmitted 
from the transmitting unit and corresponding to the position 
information and the object information; and a step S31, 
shown in FIG. 10, as one example of a display control unit 
for controlling display of an image produced by combining 
the attached information received by the receiving unit and 
the image information taken in by the image input unit. 

[0056] Also, an information processing apparatus (e.g., a 
server) according to the first embodiment of the present 



invention includes a step S8, shown in FIG. 9, as one 
example of a receiving unit for receiving the attached 
information to be attached to the real world, the position 
information representing a position in the real world, and the 
object information for identifying an object in the real world, 
these three types of information being transmitted from 
another information processing apparatus (e.g., a user ter- 
minal); a database 12a, shown in FIG. 3, as one example of 
a storage unit for storing the attached information received 
by the receiving unit in correspondence to the position 
information and the object information; a step S27, shown in 
FIG. 10, as one example of a reading unit for reading, from 
the storage unit, the attached information corresponding to 
the position information and the object information; and a 
step S28, shown in FIG. 10, as one example of a transmit- 
ting unit for transmitting the attached information read by 
the reading unit. 

[0057] An AR system according to the first embodiment of 
the present invention differs from the conventional AR 
system in that information flows bi-directionally and is not 
merely "augmented/* In view of the above meaning, the 
present AR system is referred to as an Augment-able Reality 
system hereinafter. Also, in the following description, con- 
ventional desktop, laptop, and notebook computers are col- 
lectively referred to as normal computers for the purpose of 
clear distinction from wearable computers. FIG. 2 illustrates 
the concept of Augment-able Reality according to the first 
embodiment of the present invention. In the first embodi- 
ment, a wearable computer 2 that the user can wear on 
himself or herself is employed as one example of a user 
terminal. Of course, the user terminal may be, e.g., a 
portable computer instead of a wearable computer. The 
wearable computer 2 is always held in an ON-state to 
observe situations in the real world environments 1 and to 
present appropriate information to the user as required. 

[0058] A sensor 3 includes, e.g., an IR (infrared) sensor, a 
CCD video camera and a microphone to take in context 
information (position information, ID information, voice 
information, etc.) of the real world environments 1. The 
context information taken in by the sensor 3 is supplied to 
a server 5 via a network. The server 5 reads data, which 
corresponds to the supplied context information, from an 
internal database 5a in which data corresponding to the 
context information is stored beforehand, and renders the 
read data corresponding to the supplied context information 
to be supplied to and displayed on an HUD 4 via a network. 
With such a system, the user having the wearable computer 
2 put on himself or herself can acquire the data correspond- 
ing to the situations in the real world environments 1. 

[0059] Also, the user having the wearable computer 2 put 
on himself or herself creates, with the aid of the sensor 3, 
image data, text data, voice data, etc. which are to be 
attached (or affixed) to the situations in the real world 
environments 1. The attached data thus created is supplied to 
the server 5 via the network. The server 5 stores the supplied 
attached data in the database Sa in correspondence to the 
context information of the real world environments 1, The 
server 5 also renders the attached data, which has been 
newly stored in the database, to be displayed on the HUD 4 
via the network as required. 

[0060] When another user having a wearable computer 8 
put on himself or herself is in the same situation as the user 
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having the wearable computer 2 put on himself or herself, 
the server 5 presents via the network the data corresponding 
to the situation to the other user having the wearable 
computer 8 as well, 

[0061] The information corresponding to the situations in 
the real world can also be attached and browsed from normal 
computer environments. For example, when the user having 
the wearable computer 2 attaches information to a certain 
room, the attached information can be accessed from a 
normal computer 6 through, e.g., a map on the Web. When 
another user transmits an E-mail addressed to a "meeting 
room A" from a normal computer 7, the server 5 stores the 
received E-mail in the internal database 5a. Then, when the 
user having the wearable computer 2 comes to the "meeting 
room A," the contents of the E-mail sent to the "meeting 
room A" are given to that user. 

[0062] A configuration of the Augment-able Reality sys- 
tem according to the first embodiment will be described 
below with reference to FIG. 3. It is assumed that the 
Augment-able Reality system shown in FIG. 3 is installed 
in a building. 

[0063] A wearable unit 11 mainly includes the CCD video 
camera 21, the IR sensor 22, a mouse 23, the microphone 24, 
an interface box 25, a sub-notebook PC (Personal Com- 
puter) 26, an HUD 27, and an earphone 28 which are all 
carried with the user. The wearable unit 11 (the sub-note- 
book PC 26) is connected to a server 12 via a wireless LAN 
(Local Area Network) 16. The server 12 includes the data- 
base 12a therein. A normal computer 41 is connected to the 
server 12 via a WWW (World Wide Web) interface 13. A 
normal computer 42 is connected to the server 12 via an 
E-mail interface 14. 

[0064] The Augment-able Reality system shown in FIG. 3 
employs, e.g., both an IR beacon 17 and a visual ID for 
recognizing situations (such as a position and an object) in 
the real world. The IR beacon 17 is installed in each room 
of the building, and transmits an IR signal at intervals of 
predetermined time. This IR signal enables a position rec- 
ognition to be made at a room level. Also, recognition at an 
object level can be made by pasting a visual ID 52, which 
is formed by printing, e.g., a two-dimensional bar code 
(so-called cybercode) shown in FIG. 4B, onto an object 
such as a video cassette recorder (VCR) 51 shown in FIG. 
4A. 

[0065] The CCD video camera 21 takes in an image of the 
real world. The CCD video camera 21 also takes in two- 
dimensional bar code data of the visual ID 52 pasted onto the 
object 51. The image data and the two-dimensional bar code 
data of the visual ID 52 thus taken in are supplied to the 
sub-notebook PC 26 via a PCMCIA (Personal Computer 
Memory Card International Association) interface 31. 

[0066] The IR sensor 22 receives the IR signal transmitted 
from the IR beacon 17, and then supplies the received signal 
to the sub-notebook PC 26 via the interface box 25. 

[0067] The user operates the mouse 23 to input informa- 
tion to the sub-notebook PC 26. Also, when the user inputs 
a voice through the microphone 24, the inputted voice data 
is supplied to the sub-notebook PC 26. 

[0068] The sub-notebook PC 26 will be described in more 
detail with reference to FIG. 5. 



[0069] The image data and the two-dimensional bar code 
data taken in by the CCD video camera 21 are supplied to 
the image processing unit 61 via the PCMCIA interface 31. 
The image processing unit 61 executes a predetermined 
process on the supplied image data, and supplies the pro- 
cessed image data to a control unit 65. Further, the image 
processing unit 61 recognizes the supplied two-dimensional 
bar code data, identifies the object 51, and supplies the 
object data to the control unit 65. 

[0070] The signal received by the IR sensor 22 is supplied 
to a position information detecting unit 62 via the interface 
box 25. The position information detecting unit 62 detects 
the position of a room (identifies a room) based on the 
supplied signal, and supplies the detected position data to the 
control unit 65. 

[0071] The data inputted through the mouse 23 is supplied 
to the control unit 65. The voice data inputted through the 
microphone 24 is supplied to a voice processing unit 63. The 
voice processing unit 63 executes a predetermined process 
on the supplied voice data, and supplies the processed voice 
data to the control unit 65. 

[0072] The control unit 65 transmits the supplied position 
data, object data, image data and voice data to the server 12 
via a communication unit 32. The transmitted data is 
received via the wireless LAN 16 by a communication unit 
(not shown) incorporated into the server 12. A combining 
unit 66 combines an output of the CCD video camera 21 and 
an output of the control unit 65 with each other, and supplies 
the combined output to the HUD 27 for display on it. 

[0073] The server 12 stores the received image data and 
voice data in the database 12a in correspondence to the 
position information and the object information. 

[0074] The process of reading the information attached to 
the situations of the real world in the first embodiment will 
be explained below. 

[0075] When the position data and the object data are 
supplied respectively from the position information detect- 
ing unit 62 and the image processing unit 61, the control unit 
65 transmits the position data and the object data to the 
server 12 through the communication unit 32. The transmit- 
ted position data and object data are received via the wireless 
LAN 16 by the communication unit incorporated in the 
server 12. 

[0076] The server 12 reads, from the database 12a, the 
image data and the voice data corresponding to the trans- 
mitted position data and object data. The read image data 
and voice data are transmitted to the sub -notebook PC 26 via 
the communication unit. 

[0077] The transmitted image data and voice data are 
received by the communication unit 32 of the sub-notebook 
PC 26 via the wireless LAN 16. The communication unit 32 
supplies the received image data and voice data to the 
control unit 65. The control unit 65 creates icons corre- 
sponding to the supplied image data and voice data, respec- 
tively, and supplies the created icons to the combining unit 
66. The control unit 65 also outputs the supplied voice data 
through the earphone 28. The combining unit 66 combines 
the corresponding icon with the supplied image data of the 
current sight, followed by displaying a combined image on 
the HUD 27. As a result, the user having the wearable unit 
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11 put on himself or herself can acquire information corre- 
sponding to the situations of the real world. 

[0078] One example of a display screen image on the 
HUD 27 will be described below with reference to FIG. 6. 
The following example of a display screen image represents 
the case that the user operates a VCR (object) at a location 
of an editing studio and pastes, as a voice memo, a point 
found out by the user about the VCR (object). 

[0079] As shown in FIG. 6, the display screen image on 
the HUD 27 is divided into a personal information tray 71 
and a context-aware area 72. In the personal information tray 
71, information carried with the user, such as a voice memo 
and an image snap shot, is indicated by a voice icon memo 
75. 

[0080] The context-aware area 72 corresponds to the situ- 
ation of the real world, and the screen image in the area 72 
varies each time the situation is changed with movement of 
the user. The context- aware area 72 is divided into a location 
field 73 corresponding to the position information at a room 
level, and an object field 74 corresponding to the recognized 
object. When the user is present in the editing studio and 
stands in front of a VCR, an icon 76 representing that the 
user is now present in the editing studio is indicated on the 
left side of the location field 73, and an icon 77 representing 
that the user is now standing in front of the VCR is indicated 
on the left side of the object field. In the location field 73, a 
voice memo that is information attached to the editing studio 
is represented as an icon 78. By clicking the icon 78 with the 
mouse 23, the user can listen the corresponding voice 
through the earphone 28. In the object field 74, information 
attached to the VCR is represented as a voice memo icon 79 
and an image snap shot icon 80. 

[0081] One example of screen images displayed when the 
user attaches information created in the personal information 
tray 71 to the context-aware area 72, will be described below 
with reference to FIGS. 7 A to 7C. 

[0082] When the user clicks a RECORD button 84 with 
the mouse 23, a voice note icon 91 is produced (FIG. 7A) 
and is displayed in the personal information tray 71 (FIG. 
7B). Then, when the user operates the mouse 23 to drag and 
drop the icon 91 in the location field 73 for the editing 
studio, the icon 91 is pasted to the room of the editing studio 
(FIG, 7C), The result of the above operation is transmitted 
to the server 12 via the wireless LAN 16. The server 12 
registers the received data in the database 12a and provides 
the data to other users as required. 

[0083] One example of a screen image in a time machine 
mode will be described below with reference to FIG. 8. If 
not specified otherwise, the following description is made in 
connection with the case that today is April 2 and the user 
goes back one day, i.e., to April 1 and returns to the front of 
the VCR in the meeting room. 

[0084] An icon 101 representing that the user is in the 
meeting room on April 1 is indicated on the left side of the 
location field 73. By clicking a context switch button 107 on 
the left side of the icon 101, the user can go back to the 
rooms which the user has visited so far. By clicking a context 
switch button 108 on the left side of the icon 102, the user 
can go back to the objects (things) which have been recorded 
so far. 



[0085] A time indicator 109 indicates the date and time of 
the situation displayed in each field. In this case, it is seen 
from the time indicator 109 that the meeting room at 12:15 
one day before, i.e., on April 1, is displayed in the location 
field 73, and the VCR at 12:15 on April 1 is displayed in the 
object field 74. Further, it is seen from the time indicator in 
the personal information tray 71 that the current date and 
time is 12:15 on April 2. By clicking up and down buttons 
(U, D) in each field, the user can move to the situation in the 
future or past. For example, by pushing an up button U73 in 
the time indicator 109 displayed in the location field 73, the 
user can move to the meeting room on April 3 (tomorrow), 
for example, and attach information to that situation. 

[0086] Icons 103 to 106 represent items of information 
which are attached to the respective fields. 

[0087] The process of creating information to be attached 
to situations in the real world and registering the information 
in the database 12a, which is prepared on the network, in the 
Augment-able Reality system shown in FIG. 3 will be 
described below with reference to a flowchart of FIG. 9. 

[0088] First, when the user having the wearable unit 11 put 
on himself or herself moves to a predetermined room (e.g., 
the editing studio), the IR sensor 22 receives the IR signal 
transmitted from the IR beacon 17 and supplies the received 
signal to the position information detecting unit 62 via the 
interface box 25 in step SI. 

[0089] In step S2, the position information detecting unit 
62 detects, based on the supplied signal, the position at a 
room level (e.g., the fact that the room is the editing studio), 
and supplies the detected position data to the control unit 65. 

[0090] In step S3, the CCD video camera 21 takes in the 
visual ID (e.g., two-dimensional bar code) 52 pasted onto 
the object (e.g., VCR) 51, and supplies the inputted visual ID 
52 to the image processing unit 61 via the PCMCIA interface 
31. 

[0091] In step S4, the image processing unit 61 identifies 
the object (e.g., VCR) 51 based on the supplied visual ID 52, 
and supplies the identified object data to the control unit 65. 

[0092] In step S5, the user having the wearable unit 11 put 
on himself or herself inputs through the microphone 24 a 
voice (i.e., a voice memo) to be attached to the editing studio 
and the VCR, and the inputted voice data is supplied to the 
voice processing unit 63. 

[0093] In step S6, the voice processing unit 63 executes a 
predetermined process on the supplied voice data and sup- 
plies the processed voice data to the control unit 65, fol- 
lowed by proceeding to step S7. 

[0094] In step S7, the control unit 65 transmits the sup- 
plied position data, object data and voice data to the server 
12 via the communication unit 32, followed by proceeding 
to step S8. 

[0095] In step S8, the communication unit (not shown) of 
the server 12 receives the transmitted position data, object 
data and voice data via the wireless LAN 16. 

[0096] In step S9, the server 12 registers the received 
voice data in the database 12a in correspondence to the 
position data and the object data, thereby completing the 
process. 
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[0097] The process of reading and displaying the infor- 
mation, which has been attached to situations in the real 
world, from the database 12a in the Augment-able Reality 
system shown in FIG. 3 will be described below with 
reference to a flowchart of FIG. 10. 

[0098] First, when the user having the wearable unit 11 put 
on himself or herself moves to a predetermined room (e.g., 
the editing studio), the IR sensor 22 receives the IR signal 
transmitted from the IR beacon 17 and supplies the received 
signal to the position information detecting unit 62 via the 
interface box 25 in step S21, 

[0099] In step S22, the position information detecting unit 
62 detects, based on the supplied signal, the position at a 
room level (e.g., the fact that the room is the editing studio), 
and supplies the detected position data to the control unit 65. 

[0100] In step S23, the CCD video camera 21 takes in the 
visual ID (e.g., two-dimensional bar code) 52 pasted onto 
the object (e.g., VCR) 51, and supplies the inputted visual ID 
52 to the image processing unit 61 via the PCM CIA interface 
31. 

[0101] In step S24, the image processing unit 61 identifies 
the object (e.g., VCR) 51 based on the supplied visual ID 52, 
and supplies the identified object data to the control unit 65. 

[0102] In step S25, the control unit 65 transmits the 
supplied position data and object data to the server 12 via the 
communication unit 32, followed by proceeding to step S26. 

[0103] In step S26, the communication unit (not shown) of 
the server 12 receives the transmitted position data and 
object data via the wireless LAN 16. 

[0104] In step S27, the server 12 reads, from the database 
12a, the voice data corresponding to the received position 
data and object data, followed by proceeding to step S28. 

[0105] In step S28, the server 12 transmits the read voice 
data to the sub-notebook PC 26 from the transmission unit, 
followed by proceeding to step S29. 

[0106] In step S29, the transmission unit 32 of the sub- 
notebook PC 26 receives the transmitted voice data via the 
wireless LAN 16, and supplies the received voice data to the 
control unit 65, followed by proceeding to step S30. 

[0107] In step S30, the control unit 65 produces an icon 
corresponding to the supplied voice data, and supplies the 
created icon to the combining unit 66. 

[0108] In step S31, the combining unit 66 combines the 
created icon with the image data of the current sight supplied 
from the CCD video camera 21, and displays a combined 
image on the HUD 27, thereby completing the process. 

[0109] The case of accessing the database 12a from nor- 
mal computer environments will be described below. FIG. 
11 shows one example of a screen image when information 
of the database 12a is acquired from a map on the WWW 
described by, e.g., the so-called Java applet. In the screen 
image of FIG. 11, the user can also attach information to 
situations (such as a position and an object) in the real world 
acquired via the WWW and displayed on the screen. 

[0110] Further, the user can send an E-mail addressed to 
the situation of the real world by utilizing the computer 42. 
For example, when an E-mail is transmitted from the com- 
puter 42 to a "room 3" as shown in FIG. 12, the server 12 



receives the E-main via the E-mail interface 14. Then, when 
another user having the wearable unit 11 put on himself or 
herself comes to the "room 3," the contents of a message 
(e.g., "Today's meeting is cancelled.") is displayed on the 
HUD 27 and informed to the other user. 

[0111] While an IR beacon is employed to detect a posi- 
tion in the above first embodiment, detection of the position 
can also be made by utilizing the GPS (Global Positioning 
System) or PHS (Personal Handyphone System). 

[0112] Additionally, a providing medium for providing a 
computer program for executing the above processes in the 
first embodiment to users includes not only an information 
recording medium such as a magnetic disk or CD-ROM, but 
also a transmission medium in the form of a network 
employing the Internet or digital satellites. 

[0113] It is to be noted that the term "system" used in the 
first embodiment implies an overall facility made up of a 
number of devices, units and the like. 

[0114] Next, a second embodiment of the present inven- 
tion will be described with reference to FIGS. 13 to 20. 

[0115] The second embodiment of the present invention 
relates to an information processing system in which an ID 
already attached to an object in the real world is read by an 
ID reader so that, based on the read ID, the user can register 
and retrieve a bulletin board in the BBS, for example, on the 
Internet or PC communication services, can browse infor- 
mation of the object corresponding to the ID on the retrieved 
bulletin board, and can write and exchange information of 
the object to which the ID is attached. 

[0116] FIG. 13 shows one concrete system according to 
the second embodiment of the present invention. In this 
embodiment, a wearable sub-notebook PC is used as one 
example of a personal computer. The system shown in FIG. 
13 represents the case of employing a client PC 304 as one 
example of an information processing apparatus (i.e., a user 
terminal), a book 307 as one example of an object in the real 
world, a bar code 306 as one example of an ID attached to 
the book 307, a bar code reader 305 for reading the bar code 
306 as one example of an ID reader for reading the ID, the 
Internet as one example of a network 303, and a server 301 
including a hard disk 302 as one example of another 
information processing apparatus, respectively. The con- 
figuration of FIG. 13 is shown by way of example and the 
present invention is not limited to the illustrated one. 

[0117] In FIG. 13, the server 301 is similar to an ordinary 
WWW (World Wide Web) server, and comprises a main unit 
computer (server 301), the hard disk 302 for storing infor- 
mation of bulletin boards associated with objects corre- 
sponding to IDs, a network connecting device (not shown) 
for connecting the server 301 to the network 303, etc. 

[0118] Further, the client PC 304 comprises a sub-note- 
book PC including a modem, a PCMCIA interface, etc., and 
is connectable to the Internet or the like via the modem. The 
client PC 304 is also connectable to the bar code reader 305 
(possibly a CCD camera instead), which serves as the ID 
reader, via the PCMCIA interface. The device for reading 
the ID is not limited to an independent standalone device 
such as the ID reader in FIG. 13 (e.g., the bar code reader), 
but may be such a device as assembled in the client PC 304. 
For example, in the case that a CCD camera is assembled in 
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the client PC 304, the CCD camera can be used as the ID 
reader. As another example, when numerals or the like that 
are visually readable by the user are employed as an ID 
attached to an object in the real world, a keyboard (e.g., a 
keyboard of the client PC 304) for entering the ID is also 
included in the category of the ID reader. 

[0119] FIG. 14 shows one example of an initial screen 
image displayed on a display unit (e.g., a liquid crystal 
display) of the client PC 304 when an ID (e.g., the bar code 
306) already attached to an article (e.g., the book 307) in the 
real world is taken in by an ID reader (e.g., the bar code 
reader 305) and bulletin boards present in the Internet server 
301, for example, are searched based on the inputted ID. 
Note that FIG. 14 shows one example of the initial screen 
image and the present invention is not limited to the illus- 
trated one. 

[0120] Referring to FIG. 14, an ID entry field 311 is 
indicated in the initial screen image. When the user wants to 
browse the bulletin board which carries information regard- 
ing the book 307 in the real world, such as critics and details, 
the user operates the bar code reader 305 to read the bar code 
306 attached to the book 307 (or the user enters numerals of 
the bar code from the keyboard), whereupon the numerals 
corresponding to the bar code 306 are inputted in the ID 
entry field 311 of the initial screen image. 

[0121] When the ID (numerals) is inputted in the ID entry 
field 311, the client PC 304 accesses the server 301 in such 
a manner as to encode the ID into a URL (Uniform Resource 
Locator) of the WWW. If the ID is recognized as, e.g., 
"9780123456789," a URL is given below, for example: 

http ://www.aaa.com/cgi-bin/bbs . cgi? id= 
9780123456789 

[0122] If the bulletin board corresponding to the ID is 
already registered in the server 301 upon such an access to 
the server 301, the server 301 displays an image of the 
bulletin board corresponding to the ID as shown in FIG. 15, 
by way of example, on the display screen of the client PC 
304. Note that FIG. 15 shows one example of the display 
screen image and the present invention is not limited to the 
illustrated one. 

[0123] More specifically, the screen image of the bulletin 
board shown in FIG. 15 includes, e.g., an ID number field 
312 for indicating the ID number ("9780123456789"); a 
bulletin board name field 313 for indicating the name of the 
bulletin board ("Bulletin board of book 'AABBCC'" in this 
example); a link address field 314 for link to another search 
service (e.g., a link for on-line shopping); a mailing field 315 
in which a field, buttons, etc. for new mailing to the bulletin 
board are indicated; and a registered information field 316 
for indicating information already registered on the bulletin 
board. Since the bulletin board in the format shown in FIG. 
15 enables users to browse and exchange information of the 
object (book) corresponding to the ID, each user can refer to 
opinions of others, retrieve a desired bulletin board, or write 
new information on the bulletin board as with the ordinary 
BBS, for example. 

[0124] On the other hand, in some cases, when the server 
301 is accessed with a URL corresponding to the ID, the ID 
is not yet registered in the server 301 (the bulletin board 
corresponding to the ID is not present). 

[0125] In such a case, the server 301 displays, on the 
display screen of the client PC 304, a new registration screen 



image for prompting the user to make new registration of a 
bulletin board corresponding to the ID as shown in FIG. 16, 
by way of example. Note that FIG. 16 shows one example 
of the display screen image and the present invention is not 
limited to the illustrated one. 

[0126] More specifically, the screen image for new regis- 
tration shown in FIG. 16 includes a name entry field 317 for 
prompting the user to enter basic information, e.g., the name 
of the article, corresponding to the ID ("9881234" in the 
example of FIG. 16). When the user inputs, in the name 
entry field 317, the name of the article ("Health Drink XYZ" 
in the example of FIG. 16) corresponding to the ID, a new 
bulletin board designated by the ID ("9881234") and 
"Health Drink XYZ," i.e., the name of the article corre- 
sponding to the ID, is registered in the server 301. 

[0127] With the second embodiment of the present inven- 
tion, as described above, based on the ID already attached to 
an article (object) in the real world, the user can register and 
retrieve, e.g., a bulletin board in the Internet BBS, can 
browse information of the article corresponding to the ID on 
the retrieved bulletin board, and can write and exchange 
information of the article corresponding to the ID. 

[0128] Of course, the second embodiment is not limited to 
the book or health drink described above, but is also 
applicable to other suitable products. By scanning a bar code 
attached to a music CD, for example, it is possible to access 
or jump to a bulletin board for the CD, and hence to easily 
acquire information regarding the music of the CD. Since 
the bulletin board for the CD can be accessed by other users 
who are interested in the same CD, circulation of informa- 
tion and formation of user communities is promoted. Like- 
wise, by scanning a bar code attached to an electric product, 
for example, it is possible to obtain the latest information 
relating to the electric product from the user community. 

[0129] In the system of the second embodiment, the infor- 
mation written on the bulletin board is not limited to 
information provided from the client PC side as described 
above, but may be, e.g., information of various products 
marketed from the product selling side. When the product 
selling side offers the product information via the BBS, a 
place for exchanging information can be provided through a 
universal interface for, e.g., all types of products marketed 
from the product selling side. 

[0130] In the second embodiment, the ID is assumed to be 
a typical bar code as described above. This is because model 
numbers are printed or attached in the form of bar codes onto 
most of products and they can be used as IDs as they are. 
There are other available ID systems including, e.g., the 
JAN code (the EAN code in Europe), the ISBN code, and the 
CDDB ID. The JAN code is one type of bar code systems, 
and is printed in the form of a bar code on a product package 
for uniquely identifying a packaged product. The ISBN code 
is a code system for uniquely identifying a book, and the 
code number is printed on, e.g., a back cover of each book. 
In many cases, a bar code corresponding to the ISBN code 
is also printed along with the ISBN code. The CDDB ID is 
an ID system used on the Internet for identifying CDs, and 
is already available at the URL/http:/www.cddb.com. More 
specifically, the CDDB ID is created by processing infor- 
mation of a music CD, such as the number of tracks and the 
playing time, for conversion into numerals of 32 bits. The 
CDDB ID number can be used as an ID of each CD. Further, 
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the name capable of uniquely identifying an object can also 
be used as an ID in addition to a bar code. For example, the 
address of E-mail for identifying persons, the URL of a 
home page on the WWW, the phone number, the address, 
and other proper names such as the company name, the trade 
name, the name of a store and the brand name, can be used 
as an ID. 

[0131] Additionally, it is conceivable that a two-dimen- 
sional ID such as the so-called cybercode and a non-contact 
ID such as a wireless tag are used as IDs. Depending upon 
the type of ID, the screen image can be linked with another 
information service, e.g., on-line shopping. The example 
shown in FIG. 15 represents a BBS screen image provided 
when scanning the universal number (e.g., the ISBN code) 
of a book with a bar code reader, and includes not only 
information of the bulletin board regarding the book, but 
also the link address, indicated in the link address field 314, 
to an on-line book selling page. 

[0132] FIG. 17 shows a process flow when on the client 
PC side in the system according to the second embodiment, 
an ID attached to an article in the real world is read by an 
ID reader, and retrieval and browsing of and writing on a 
bulletin board corresponding to the ID are performed, or a 
bulletin board is newly registered. Note that FIG. 17 shows 
one example of the process flow and the present invention is 
not limited to the illustrated one. 

[0133] Referring to FIG. 17, in step S41, the client PC 304 
first starts application software for running a BBS search 
with an ID according to the second embodiment. In the 
second embodiment, since the client PC 304 including a 
personal computer is employed as one example of the user 
terminal, the application software is started to run a BBS 
search in step S41. However, if the user terminal is a 
dedicated device for exclusively executing a BBS search and 
display with an ID, the dedicated device is powered on in 
step S41. 

[0134] When the application software is started in step 
S41, the client PC 304 waits in step S42 for an input of an 
article ID from the ID reader 305. If the article ID 306 is 
inputted from the ID reader 305, the client PC 304 estab- 
lishes connection to the server 301 in step S43, and transmits 
the inputted ID 306 to the server 301 in step S44 after 
encoding the ID into a corresponding URL. 

[0135] Then, in step S45, the client PC 304 receives a 
reply from the server 301. The reply from the server 301 is 
provided as either information of a bulletin board corre- 
sponding to the ID as shown in FIG. 15, or a request for 
registration of a new bulletin board (registration of a new 
ID) as shown in FIG. 16. Accordingly, the processing in the 
client PC 304 advances to steps subsequent to S47 if the 
reply from the server 301 is information of a bulletin board, 
and to steps subsequent to S52 if the reply from the server 
301 is a request for registration of a new bulletin board. 

[0136] If it is determined in step S46 that the reply from 
the server 301 is information of a bulletin board, the client 
PC 304 displays an image screen of the bulletin board 
corresponding to the ID as shown in FIG. 15, by way of 
example, on the display screen in step S47. 

[0137] On the other hand, if it is determined in step S46 
that the reply from the server 301 is a request for registration 
of a new bulletin board, the client PC 304 goes to step S52 



in which it displays, on the display screen, a new registration 
screen image for prompting the user to make new registra- 
tion of a bulletin board corresponding to the ID as shown in 
FIG. 16, and also indicates, in the screen image for new 
registration, the name entry field 317 for prompting the user 
to enter the name of an article corresponding to the ID. 

[0138] Thereafter, in step S53, the client PC 304 transmits 
the name of the article entered in the name entry field 317 
to the server 301. After the completion of step S53, the 
process flow returns to step S45. As a result, the newly 
registered bulletin board is transmitted to the client PC 304 
from the server 301. 

[0139] Then, the client PC 304 determines in step S48 
whether there is any information to be written on the bulletin 
board. If there is any information to be written on the 
bulletin board, the client PC 304 transmits the written 
information to the server 301 in step S49. After the comple- 
tion of step S49, the process flow goes to step S50. 

[0140] If it is determined in step S48 that there is no 
information to be written on the bulletin board, the process- 
ing in the client PC 304 advances to step S50. 

[0141] In step S50, the client PC 304 determines whether 
there is an input of a new ID from the ID reader. If it is 
determined in step S50 that a new ID is inputted from 
another article, for example, the processing of the client PC 
304 returns to step S44, following which the new ID is 
transmitted to the server 301. On the other hand, if it is 
determined in step S50 that a new ID is not inputted, the 
process flow goes to step S51. 

[0142] If an input of a new ID is not determined in step 
S50 and cutoff of the communication line is instructed from 
the user, the client PC 304 cuts off the communication line 
between itself and the server 301 in step S51. After cutoff of 
the communication line in step S51, if it is determined in 
step S54 that the application has been brought into end or the 
power has been turned off, the processing shown in the 
flowchart of FIG. 17 is ended. If it is determined in step S54 
that neither the application has been brought into end nor the 
power has been turned off, the process flow returns to step 
S42 in which the client PC 304 waits for an ID input. In the 
flowchart of FIG. 17, the communication line between the 
server 301 and the client PC 304 is connected at the time of 
starting transmission of the ID to the server 301, and the 
communication line is cut off upon an instruction from the 
user to cut off it. However, the client PC 304 may be kept 
always connected to the server 301. 

[0143] FIG. 18 shows a process flow when, in response to 
an ID transmitted from the client PC 304, the bulletin board 
corresponding to the ID or a request for new registration of 
a bulletin board is transmitted to the client PC 304 from the 
side of the server 301 in the second embodiment. Note that 
FIG. 18 shows one example of the process flow and the 
present invention is not limited to the illustrated one. 

[0144] Referring to FIG. 18, as shown in step S61, the 
server 301 is always held in a state waiting for reception of 
an ID from the client PC 304. If the server 301 receives an 
ID from the client PC 304 in step S61, the process flow goes 
to step S62. 

[0145] After receiving an ID from the client PC 304 in step 
S61, the server 301 determines in step S62 whether the 



06/22/2004, EAST version: 1.4.1 



US 2004/0056902 Al 



10 



Mar. 25, 2004 



received ID is registered, i.e., whether the bulletin board 
corresponding to the received ID is registered. If it is 
determined in step S62 that the bulletin board corresponding 
to the received ID is not registered, the processing in the 
server 301 advances to steps subsequent to S66, whereas if 
it is determined in step S62 that the bulletin board corre- 
sponding to the received ID is registered, the processing in 
the server 301 advances to steps subsequent to S63. 

[0146] If it is determined in step S62 that the bulletin 
board corresponding to the received ID is registered, the 
server 301 transmits information of the bulletin board cor- 
responding to the ID to the client PC 304 in step S63. After 
the completion of step S63, the process flow goes to step 
S64. 

[0147] On the other hand, it is determined in step S62 that 
the bulletin board corresponding to the received ID is not yet 
registered, the server 301 transmits, to the client PC 304, 
information of the new registration screen image for prompt- 
ing the user to make new registration of the bulletin board 
corresponding to the ID as shown in FIG. 16. 

[0148] After receiving the name of an article correspond- 
ing to the ID from the client PC 304 in step S67, the server 
301 creates and registers, as the bulletin board correspond- 
ing to the received ID, a bulletin board for the name of the 
article in step S68, and then transmits information of the 
newly registered bulletin board to the client PC 304 in step 
S69. After the completion of step S69, the process flow goes 
to step S64. 

[0149] Then, the server 301 determines in step S64 
whether new information is written on the bulletin board 
from the client PC 304, i.e., whether new written informa- 
tion is transmitted from the client PC 304. If it is determined 
in step S64 that no information is written on the bulletin 
board, the processing in the server 301 returns to step S61 
in which the server 301 is held in the state waiting for 
reception of an ID from the client PC 304. 

[0150] On the other hand, if it is determined in step S64 
that information is written on the bulletin board, the server 
301 adds the written information to the bulletin board or 
updates the information on the bulletin board depending 
upon the written information in step S65. After the comple- 
tion of step S64, the processing in the server 301 returns to 
step S61 in which the server 301 is held in the state waiting 
for reception of an ID from the client PC 304. 

[0151] The above second embodiment has been described 
in connection with an example in which one bulletin board 
corresponds to one ID. However, when a number of ID 
systems including, e.g., the JAN code (the EAN code in 
Europe), the ISBN code, the CDDB ID, and wireless tags, as 
described above, are usable together in the information 
processing system, it may happen that the same ID is used 
among a number of different ID systems. In such a case, one 
ID is used double or more and represents a number of 
different articles. If a number of different articles have the 
same ID, this results in that when a bulletin board corre- 
sponds to that ID is retrieved, a number of bulletin boards for 
the number of different articles are found out by a search. 

[0152] In view of the above, the system of this embodi- 
ment is designed such that when a number of bulletin boards 
are found corresponding to the same ID in the server 301, the 
names, abstracts, etc. of the number of bulletin boards are 



provided to the client PC 304 so that the user may select 
desired one from among the number of bulletin boards. 

[0153] FIG, 19 shows a process flow on the side of the 
server when a number of bulletin boards corresponding to 
the same ID are found out by a search as described above. 
Concretely, the process flow of FIG. 19 can be effected in, 
e.g., step S63 in the flowchart of FIG. 18. Note that FIG. 19 
shows one example of the process flow and the present 
invention is not limited to the illustrated one. 

[0154] Referring to FIG. 19, in step S71, the server 301 
first determines whether when an ID is transmitted from the 
client PC 304, a number of bulletin boards are registered 
corresponding to the ID. 

[0155] If it is determined in step S71 that a number of 
bulletin boards are not registered corresponding to the ID, 
information of the bulletin board corresponding to the ID is 
transmitted to the client PC 304 in step S74. 

[0156] On the other hand, if it is determined in step S71 
that a number of bulletin boards are registered correspond- 
ing to the ID, the server 301 transmits, e.g., a list of the 
names, abstracts, etc. of the number of bulletin boards to the 
client PC 304 in step S72. 

[0157] After transmitting the list of the names, abstracts, 
etc. of the number of bulletin boards to the client PC 304, 
upon receiving a selection signal for the name, etc. (i.e., a 
selection signal for the bulletin board) which has been 
selected by the client PC 304 from among the list of the 
names, abstracts, etc. of the number of bulletin boards in 
step S73, the server 301 transmits information of the bulletin 
board corresponding to the selection signal to the client PC 
304 in step S74. 

[0158] FIG. 20 shows a process flow on the side of the 
client PC 304 when the list of the names, abstracts, etc. of 
the number of bulletin boards are transmitted from the side 
of the server 301. Concretely, the process flow of FIG. 20 
can be effected in, e.g., step S47 in the flowchart of FIG. 17. 
Note that FIG. 20 shows one example of the process flow 
and the present invention is not limited to the illustrated one. 

[0159] Referring to FIG. 20, in step S81, the client PC 304 
first determines whether the signal transmitted from the 
server 301 provides the list of the names, etc. of the number 
of bulletin boards. 

[0160] If it is determined in step S81 that the signal 
transmitted from the server 301 does not provide the list of 
the names, etc. of the number of bulletin boards, i.e., that 
information of one bulletin board corresponding to the ID is 
transmitted from the server 301, the client PC 304 displays 
the only one bulletin board transmitted from the server 301 
on the display screen in step S84. 

[0161] On the other hand, if it is determined in step S81 
that the signal transmitted from the server 301 provides the 
list of the names, etc. of the number of bulletin boards, the 
client PC 304 displays the list of the names, etc. of the 
number of bulletin boards on the display screen in step S82, 
thereby prompting the user to select the desired name (i.e., 
the desired bulletin board) from among the list of the names, 
etc. 

[0162] Then, in step S83, the client PC 304 transmits to the 
server 301 the name of the bulletin board that has been 
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selected by the user from among the list of the names, etc. 
Thereafter, when information of the bulletin board corre- 
sponding to the selected name, etc. is transmitted from the 
server 301, the client PC 304 displays the bulletin board 
transmitted from the server 301 on the display screen in step 
S84. 

[0163] In the case of employing a number of different ID 
systems together, the double or more use of the same ID can 
be avoided by managing the BBS in such a manner, for 
example, that the name (e.g., ISBN) of the ID system is 
added to each ID. More specifically, when the client PC 304 
takes in an ID, it transmits the ID to the server 301 along 
with information indicating to which system the ID belongs 
(i.e., the name of the ID system). The server 301 searches 
bulletin boards based on both the ID and the information (the 
name of the ID system) added to the ID. With such a 
management, even if the same ID is employed in different 
ID systems, one bulletin board can be retrieved correspond- 
ing to one ID. 

[0164] Additionally, a providing medium for providing a 
computer program for executing the above processes in the 
second embodiment to users includes not only an informa- 
tion recording medium such as a magnetic disk or CD-ROM, 
but also a transmission medium in the form of a network 
employing the Internet or digital satellites. 

[0165] It is to be noted that the term "system" used in the 
second embodiment implies an overall facility made up of a 
number of devices, units and the like. 

[0166] Next, a third embodiment of the present invention 
will be described with reference to FIGS. 21 to 28. 

[0167] While an article in the real world (in practice, an ID 
attached to the article) is handled in correspondence to a 
bulletin board in the second embodiment, a position in the 
real world is handled in correspondence to a bulletin board 
in the third embodiment. 

[0168] Specifically, the third embodiment relates to an 
information processing system in which a current position in 
the real world is taken in by a position recognizing device, 
and a bulletin board in the BBS on the Internet or PC 
communication services, for example, can be registered 
based on the inputted current position, which enables the 
user to retrieve and browse bulletin boards corresponding to 
the inputted current position and nearby positions, and 
which enables the user to write new information or to 
exchange information on and via the bulletin boards corre- 
sponding to the inputted current position and nearby posi- 
tions. 

[0169] FIG. 21 shows one concrete system according to 
the third embodiment of the present invention. In this 
embodiment, a sub -notebook PC is used as one example of 
a user terminal and is referred to as a client PC 404 
hereinafter. The system shown in FIG. 21 represents the 
case of employing a portable client PC 404 as one example 
of an information processing apparatus (i.e., a user termi- 
nal), a position recognizing device 405 such as utilizing the 
GPS or PHS as one example of a device for detecting current 
position information in the real world, a t wireless network 
and the Internet as one example of a network 403, and a 
server 401 including a hard disk 402 as one example of 
another information processing apparatus, respectively. 
Also, the client PC 404 includes an information input device 



406 for inputting information (e.g., a text, a photograph 
and/or a voice memo) related to the current position. The 
user terminal may include the wearable computer 8 as with 
the first embodiment. The configuration of FIG. 21 is shown 
by way of example and the present invention is not limited 
to the illustrated one. 

[0170] In FIG. 21, the server 401 is similar to the WWW 
server in FIG. 13, and includes a main unit computer (server 
401), the hard disk 402 for storing information of, e.g., 
bulletin boards corresponding to respective positions in the 
real world, a network connecting device (not shown) for 
connecting the server 401 to the network 403, etc. 

[0171] The user terminal (the client PC 404) includes a 
sub-notebook PC including a modem, a PCMCIA interface, 
etc., and is connected via the PCMCIA interface to a 
wireless network device 407 (including a cellular phone and 
PHS) and the position recognizing device 405 (such as 
utilizing the GPS or PHS). Further, the client PC 404 
includes a CCD camera and a microphone as one example 
of the information input device 406 for inputting informa- 
tion related to the current position, and an internal clock for 
recognizing the current time. The CCD camera and the 
microphone serving as the information input device 406 may 
be integral with the client PC 404 or separate from the same. 
The client PC 404 can transmit current position information 
detected by the position recognizing device 405, current 
time information from the internal clock, and information 
related to the current position, such as a photograph and a 
voice memo inputted from the information input device 406 
or text data inputted from a keyboard or the like, to the 
server 401 via the wireless network device 407 and the 
network 403. 

[0172] In the system of the third embodiment, for 
example, when the bulletin board corresponding to the 
current position is not registered in the server 401 and is 
newly registered, the client PC 404 receives information 
(e.g., a photograph, a text and/or a voice mail) related to the 
current position from the information input device 406 or the 
keyboard, and transmits the received information to the 
server 401 after affixing, as search tags, the current position 
information detected by the position recognizing device 405 
and the current time information from the internal clock to 
the information related to the current position. As a result, 
the bulletin board corresponding to the current position is 
registered in the server 401, 

[0173] Also, in the system of the third embodiment, when 
the bulletin board corresponding to the current position is 
registered in the server 401 and new information is addi- 
tionally written on the bulletin board corresponding to the 
current position from the client PC 404, the client PC 404 
acquires the new information from the information input 
device 406 or the keyboard, and transmits the acquired 
information to the server 401 after affixing, as search tags, 
the current position information detected by the position 
recognizing device 405 and the current time information 
from the internal clock to the information related to the 
current position. As a result, the server 401 adds or updates 
the new information related to the current position to or on 
the bulletin board corresponding to the current position. 

[0174] Moreover, in the system of the third embodiment, 
when the bulletin board registered in the server 401 is 
retrieved or browsed from the client PC 404, the client PC 
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404 transmits to the server 401 the current position infor- 
mation detected by the position recognizing device 405 and 
additional search conditions including a keyword inputted 
from, e.g., the keyboard, and the current time information 
from the internal clock. Based on the transmitted current 
position information and additional search conditions, the 
server 401 retrieves the target bulletin board from the hard 
disk 402 and transmits information of the retrieved bulleting 
board to the client PC 404. 

[0175] FIG. 22 shows one example of a bulletin board 
transmitted from the server 401 (i.e., one example of a 
display screen image on the client PC 404) when the current 
position is in front of some restaurant having the specific 
name of "Restaurant AA," for example, and information 
instructing the name of "Restaurant AA" and the bulletin 
board being created within "7 days" from the current time is 
inputted as the additional search conditions from the client 
PC 404. The display screen image of FIG. 22 is shown by 
way of example and the present invention is not limited to 
the illustrated one. 

[0176] The display screen image of FIG. 22 includes a 
search key entry field 411 in which an additional search 
condition related to the position is inputted, and a time key 
entry field 412 in which a desired period of time from the 
current time (date and time) is inputted as another additional 
search condition. When "Restaurant AA" is inputted as a 
keyword in the search key entry field 411 and "7 days" is 
inputted as a keyword in the time key entry field 412, a 
photograph 413 of the Restaurant AA and texts 414, 415 
(e.g., "Hamburger steak is delicious" and "Spaghetti is 
unpalatable") related to comments about the Restaurant AA 
are displayed as one example of the bulletin board trans- 
mitted from the server 401 corresponding to the inputted 
keywords. 

[0177] FIG. 22 shows an example of the display screen 
image when a search is run by inputting a specific keyword, 
such as "Restaurant AA," at the current position and the 
bulletin board for "Restaurant AA" was already registered in 
the past in relation to the current position. It is however very 
rare that the current position completely coincides with the 
position at which some bulletin board was registered in the 
past. For this reason, when a search is run in the system 
according to the third embodiment of the present invention, 
bulletin boards matching the additional search conditions are 
also retrieved from among the bulletin boards which have 
been already registered at respective positions near the 
current position. In such a case, it is desired in the example 
of FIG. 22 that the distance and direction from the current 
position to "Restaurant AA" are indicated on the display 
screen image simultaneously. Further, when a search is run 
by inputting, as a specific keyword, a general name, e.g., 
"restaurant," a number of bulletin boards which are associ- 
ated with "restaurant" and have been already registered at 
respective positions near the current position are retrieved. 

[0178] FIG. 23 shows one example of a display screen 
image when bulletin boards matching with the additional 
search conditions are retrieved by the server 401 from 
among the bulletin boards corresponding to respective posi- 
tions near the current position as described above, and the 
bulletin boards corresponding to the nearby positions are 
displayed on the display screen of the client PC 404. The 



display screen image of FIG. 23 is shown by way of 
example and the present invention is not limited to the 
illustrated one. 

[0179] Referring to FIG. 23, the display screen image on 
the client PC 404 includes a distance scale 425 having the 
center O corresponding to the current position and indicating 
the distance from the center O (the current position), as well 
as the search key entry field 411 and the time key entry field 
412. In the example of FIG. 23, when a general name, e.g., 
"restaurant," is inputted as a keyword in the search key entry 
field 411 and "7 days" is inputted as a keyword in the time 
key entry field 412, bulletin boards 421, 422, 423 and 424 
matching those additional search conditions are found from 
among a number of bulletin boards which have already been 
registered at respective positions near the center O (the 
current position). The bulletin boards 421, 422, 423 and 424 
are then displayed on the display screen of the client PC 404 
as shown in FIG. 23. While the distance within the radius of 
200 m from the center O is set as a search range in the 
example of FIG. 23, the search range may be set to a larger 
or smaller distance. 

[0180] With the third embodiment of the present inven- 
tion, as described above, when the bulletin board corre- 
sponding to the current position is retrieved, a number of 
bulletin board corresponding to the current position and 
nearby positions are also obtained. Therefore, the user can 
browse, for example, opinions of other persons who visited 
the respective positions in the past. 

[0181] When running a BBS search as described above, 
filtering of bulletin boards can also be performed based on 
other attendant information (e.g., time, creator, and keyword 
explicitly attached to the bulletin board). One example of 
filtering based on time is as follows. By registering photo- 
graphs and messages about climbing of Mt. Fuji, for 
example, as a bulletin board in the server, the user can 
retrieve the bulletin board, which includes the desired pho- 
tograph, with such an additional search condition as "pros- 
pect seen from the same place in winter" when the user 
climbs Mt. Fuji in summer. This makes the user's experience 
of mountain climbing more plentiful. In streets as another 
example, the user can attach such information as "impres- 
sion of this restaurant" to the bulletin board corresponding 
to each restaurant or shop, and can browse the bulletin 
boards corresponding to the respective restaurants or shops. 
Therefore, users can form a useful set of guide book 
information while also participating themselves. 

[0182] FIGS. 24 and 25 show process flows when on the 
side of the client PC 404 in the system of the third embodi- 
ment, a bulletin board containing information related to the 
current position is newly registered, and, for example, 
retrieval and browsing of and writing on a bulletin board 
already registered corresponding to the current position are 
performed. Specifically, FIG. 24 primarily shows the pro- 
cess flow for new registration of a bulletin board, and FIG. 
25 primarily shows the process flow for, e.g., retrieval and 
browsing of and writing on a bulletin board. The process 
flows of FIGS. 24 and 25 are shown by way of example and 
the present invention is not limited to the illustrated ones. 

[0183] Referring to FIG. 24, in step S91 , the client PC 404 
first starts application software for executing registration, 
retrieval, etc. of a bulletin board corresponding to the current 
position in the third embodiment described above. In the 
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third embodiment, since a personal computer (the client PC 
404 in FIG. 21) is employed as one example of the user 
terminal, the application software is started to execute 
registration, retrieval, etc. of a bulletin board corresponding 
to the current position in step S91, However, if the user 
terminal is a dedicated device for exclusively executing 
registration, retrieval, etc. of a bulletin board corresponding 
to the current position, the dedicated device is powered on 
in step S41. 

[0184] When the application software is started in step 

591, the client PC 304 determines in step S92 whether an 
instruction from the user indicates new registration or 
retrieval of a bulletin board. If it is determined in step S92 
that retrieval of a bulletin board is instructed, the process 
flow goes to step S102 for shift to a bulletin board search 
(BBS search) routine shown in FIG. 25. On the other hand, 
if it is determined in step S92 that new registration of a 
bulletin board is instructed, the process flow goes to steps 
subsequent to S93. 

[0185] In step S93, the client PC 404 acquires or creates 
data, such as a photograph, a voice, a hand-written memo 
and/or a text, from the information input device or using the 
keyboard. The data may be created by or acquired from 
another application. 

[0186] Further, the client PC 404 acquires current position 
information from the position recognizing device in step S94 
and current time (date and time) information from the 
internal clock in step S95. Thereafter, in step S96, the client 
PC 404 adds the current position information and the current 
time information acquired respectively in steps S94 and S95, 
as search tags, to the data acquired or created in step S93. 

[0187] Then, the client PC 404 establishes a connection to 
the server 401 in step S97, and transmits the data along with 
the search tags affixed to it to the server 401 in step S98. As 
a result, the bulletin board corresponding to the current 
position is newly registered in the server 401. 

[0188] After that, the client PC 404 determines in step S99 
whether a bulletin board search is instructed from the user. 
If it is determined in step S99 that a bulletin board search is 
instructed, the process flow goes to step S102 for shift to the 
bulletin board search routine shown in FIG. 25. On the other 
hand, if it is determined in step S99 that a bulletin board 
search is not instructed, the processing in the client PC 404 
goes to step S100. 

[0189] If cutoff of the communication line is instructed 
from the user, the client PC 404 cuts off the communication 
line between itself and the server 401 in step S100. After 
cutoff of the communication line in step S100, it is deter- 
mined in step S101 whether the application has been brought 
into end or the power has been turned off. If it is determined 
that neither the application has been brought into end nor the 
power has been turned off, the process flow returns to step 

592. If it is determined that the application has been brought 
into end or the power has been turned off, the processing 
shown in the flowchart of FIG. 24 is ended. In the flowchart 
of FIG. 24, the communication line between the server 401 
and the client PC 404 is connected at the time of starting 
transmission of the data and the search tags to the server 401, 
and the communication line is cut off upon an instruction 
from the user to cut it off. However, the client PC 404 may 
be kept always connected to the server 401. 



[0190] If the process flow goes to step S102 upon such a 
determination in step S92 of FIG. 24 that new registration 
of a bulletin board is instructed, the client PC 404 acquires 
current position information from the position recognizing 
device in step Sill in FIG. 25, and then acquires, as 
additional search conditions, a keyword from, e.g., the 
keyboard and current time (date and time) information from 
the internal clock in step S112. 

[0191] Subsequently, the client PC 404 establishes a con- 
nection to the server 401 in step S113, and transmits the 
current position and the additional search conditions in step 
S114, followed by receiving a reply from the server 401 in 
step S115. 

[0192] It is here that the reply from the server 401 is 
provided as either a bulletin board retrieved based on the 
current position and the additional search conditions both 
transmitted from the client PC 404, or a message, for 
example, issued in the case that a search has ended in failure. 

[0193] Accordingly, the processing in the client PC 404 
advances to steps subsequent to S117 if it is determined in 
step S116 that the reply from the server 401 is information 
of a bulletin board, and to steps subsequent to S122 if it is 
determined in step S116 that the reply from the server 401 
is a message, for example, indicating a search failure. 

[0194] If it is determined in step S116 that the reply from 
the server 401 indicates a search failure, the client PC 404 
determines in step S122 whether a new additional search 
condition has been acquired. If it is determined that a new 
additional search condition has not been acquired, the pro- 
cessing of the client PC 404 advances to step S121. If it is 
determined that a new additional search condition has been 
acquired, the processing of the client PC 404 advances to 
step S123 in which the new additional search condition is 
transmitted to the server 401, followed by returning to step 
S115. 

[0195] On the other hand, if it is determined in step S116 
that the reply from the server 401 is information of a bulletin 
board, the client PC 404 displays one or more bulletin 
boards (corresponding to the current position or nearby 
positions as described above) transmitted from the server 
401 on the display screen in step S117. 

[0196] Then, the client PC 404 determines in step S118 
whether there is any information to be written on the bulletin 
board. If there is any information to be written, the client PC 
404 transmits the written information to the server 401 in 
step S119. After the completion of step S119, the processing 
in the client PC 404 advances to step S120. 

[0197] If it is determined in step S118 that there is no 
information to be written on the bulletin board, the process- 
ing in the client PC 404 advances to step S120. 

[0198] In step S120, the client PC 404 determines whether 
there is an input of a new current position and additional 
search condition. If it is determined in step S120 that a new 
current position and additional search condition are inputted, 
the processing in the client PC 404 returns to step S114 in 
which the new current position and additional search con- 
dition are transmitted to the server 401. On the other hand, 
if it is determined in step S120 that a new current position 
and additional search condition are not inputted, the process 
flow goes to step S12L 
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[0199] If a new current position and additional search 
condition are not inputted in step S120 and cutoff of the 
communication line is instructed from the user, the client PC 
404 cuts off the communication line between itself and the 
server 401 in step S121. After cutoff of the communication 
line in step S121, the process flow goes to step S101 in FIG. 
24. If it is determined in step S101 that the application has 
been brought into end or the power has been turned off, the 
processing is ended. If it is determined in step S101 that 
neither the application has been brought into end nor the 
power has been turned off, the process flow returns to step 
S92 in FIG. 24 in which the client PC 404 waits for an 
instruction of a retrieval or new registration of a bulletin 
board. In the flowchart of FIG. 25, the communication line 
between the server 401 and the client PC 404 is connected 
at the time of starting transmission of the current position 
and the additional search conditions to the server 401, and 
the communication line is cut off upon an instruction from 
the user to cut off it. However, the client PC 404 may be kept 
always connected to the server 401. 

[0200] FIGS. 26 and 27 show process flows when on the 
side of the server 401 in the system of the third embodiment, 
a request for retrieval of a bulletin board or data for new 
registration of a bulletin board is transmitted from the client 
PC 404, and retrieval or new registration of a bulletin board 
is run correspondingly. Specifically, FIG. 26 primarily 
shows the process flow for new registration of a bulletin 
board, and FIG. 27 primarily shows the process flow for 
retrieval of a bulletin board, etc. The process flows of FIGS. 
26 and 27 are shown by way of example and the present 
invention is not limited to the illustrated ones. 

[0201] Referring to FIG. 26, as shown in step S131, the 
server 401 is always held in a state of waiting for a request 
for retrieval of a bulletin board or data for new registration 
of a bulletin board from the client PC 404. If it is determined 
in step S131 that a request for retrieval of a bulletin board 
is transmitted from the client PC 404, the process flow goes 
to step S134 for shift to a bulletin board search routine 
shown in FIG. 27. On the other hand, if it is determined in 
step S131 that data for new registration of a bulletin board 
is transmitted from the client PC 404, the process flow goes 
to steps subsequent to S132. 

[0202] In step S132, the server 401 creates a bulletin board 
corresponding to the data, such as a photograph, a voice, a 
hand-written memo and/or a text, transmitted from the client 
PC 404, as well as the search tags, i.e., the current position 
and the current time (date and time), affixed to the data. 

[0203] Then, in step S133, the server 401 stores the 
thus-created bulletin board in the hard disk 402. The regis- 
tration process of a new bulletin board is thereby completed. 
After step S133, the server 401 returns to the state of waiting 
for reception of a signal in step S131. 

[0204] If the process flow goes to step S134 upon such a 
determination in step S131 of FIG. 26 that new registration 
of a bulletin board is instructed, the server 401 receives, in 
step S141 of FIG. 27, the current position and the additional 
search conditions both transmitted from the client PC 404. 

[0205] After receiving the current position and the addi- 
tional search conditions from the client PC 404, the server 
401 searches in step S142 the bulletin boards, which are 
stored in the hard disk 402, for one matching the current 



position and the additional search conditions. If a bulletin 
board in match with the current position and the additional 
search conditions is found out by the search in step S142, the 
processing in the server 401 advances to steps subsequent to 
S143, whereas if not found out, the processing in the server 
401 advances to steps subsequent to S146. 

[0206] If it is determined in step S142 that the search has 
finished successfully, the server 410 transmits the found-out 
bulletin board to the client PC 404 in step S143. 

[0207] Then, the server 401 determines in step S144 
whether there is any information written on the bulletin 
board from the client PC 404. If any information is written, 
the server 401 adds the written information to the bulletin 
board or updates the bulletin board with the written infor- 
mation in step S145, followed by returning to step S131 in 
FIG. 26. 

[0208] On the other hand, if it is determined in step S142 
that the search has ended in failure, the server 410 transmits 
a search failure message, for example, to the client PC 404. 
Thereafter, if a new additional search condition is received 
from the client PC 404 in step S147, the server 401 runs a 
search again in step S142 using the new additional search 
condition. If a new additional search condition is not trans- 
mitted, the process flow returns to step S131 in FIG. 26. 

[0209] When position recognition is performed by utiliz- 
ing the PHS in the third embodiment of the present inven- 
tion, one conceivable method is to calculate the latitude and 
the longitude based on the principles of triangular surveying 
by using the intensities of electric fields generated from base 
stations whose positions are known. As another method, it is 
also conceivable to employ, as a "position" of convenience, 
an ID of the base station generating an electric field with the 
maximum intensity, or to employ, as a "position" of conve- 
nience, a pair of base station ID and electric field intensity 
(possibly plural pairs depending upon cases). The reason 
why the term "of convenience" is used here is that there is 
no direct relation between the base station ID and the 
latitude and longitude. In the third embodiment of the 
present invention, since it is enough to be able to determine 
whether "positions" corresponding to registered new bulle- 
tin boards are close to each other, absolute position infor- 
mation is not necessarily required. 

[0210] In the method of employing, as a "position" of 
convenience, a pair of base station ID and electric field 
intensity, a "distance" of convenience can be calculated with 
similarity calculation made between one pair and another 
pair of base station ID and electric field intensity which is 
observed later. A computing process shown in FIG. 28, by 
way of example, can be used as a similarity determining 
algorithm for executing the similarity calculation. The com- 
puting process shown in FIG. 28 corresponds to calculation 
of correlation coefficients among a set of base station 
IDs/electric field intensities. Additionally, in FIG. 28, IDi 
represents the ID of the i-th base station, Fi represents the 
electric field intensity of the i-th base station, and d repre- 
sents the "distance" of convenience between [IDi, Fi] (i»l, 
2, .... n) and [ID'j, F'j] (j-1, 2, . . . , m). 

[0211] Referring to FIG. 28, the client PC 404 first 
initializes a variable "distance d" to zero (0) in step S151, 
and starts calculating the "distance d" of convenience 
between [IDi, Fi] (i-1, 2, .... n) and [ID'j, F'j] Q-l, 2, . . 
. , m) in step S152. 
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[0212] After starting the calculation, the client PC 404 
determines in step S153 whether a calculation loop for i, j at 
i=l, 2, . . . , n and 2, . . . , m is completed. If it is 
determined in step S153 that the calculation loop is not yet 
completed, the client PC 404 determines in step S154 
whether IDi^ID'j is satisfied. If it is determined in step S154 
that IDi=ID'j is not satisfied, the .process flow returns to step 
S153, whereas if it is determined that IDi=ID'j is satisfied, 
the process flow returns to step S153 after setting d=d+ 
Fi*Pj in step S155. 

[0213] Then, it is determined in step S153 that the calcu- 
lation loop is completed, the client PC 404 sets d=d/(m*n) 
and obtains the distance d. 

[0214] While the above-described third embodiment uti- 
lizes the GPS or PHS for detecting a position, it is also 
possible to utilize, e.g., traffic information beacons which 
have been recently installed for a traffic information pro- 
viding system, or an IR beacon used in the first embodiment 
for the same purpose. 

[0215] Additionally, a providing medium for providing a 
computer program for executing the above processes in the 
third embodiment to users includes not only an information 
recording medium such as a magnetic disk or CD-ROM, but 
also a transmission medium in the form of a network 
employing the Internet or digital satellites. 

[0216] It is to be noted that the term "system" used in the 
third embodiment implies an overall facility made up of a 
number of devices, units and the like. 

[0217] FIG. 29 shows one example of a concrete configu- 
ration of principal parts of client PCs (304, 404) applicable 
to the second and third embodiments. 

[0218] Referring to FIG. 29, an ID taken in by the ID 
reader 305 such as a bar code reader is supplied to an ID 
analyzing unit 502 via a PCMCIA interface 501. The ID 
analyzing unit 502 analyzes the supplied ID, and then 
supplies the analyzed ID data to a control unit 503. 

[0219] A position signal detected by the position recog- 
nizing device 405 such as utilizing the GPS or PHS is 
supplied to a position information computing unit 505 via a 
PCMCIA interface 504. The position information computing 
unit 505 computes, from the supplied signal, the current 
altitude and longitude (in the case utilizing the GPS) or 
position-of-convenience information (in the case utilizing 
the PHS) based on base station IDs, and then supplies the 
computed current position data to the control unit 503. 

[0220] Data such as a photograph, a voice and/or a hand- 
written memo is inputted from the information input device 
406, and the input data is supplied to an input information 
processing unit 506. The input information processing unit 
506 processes the input data depending upon the types of 
data (such as a photograph, a voice or a hand-written memo), 
and then supplies the processed data to the control unit 503. 

[0221] Other data, e.g., a text or a keyword as a search 
condition, is entered from a keyboard, etc. 508, and an 
internal clock 510 generates data of current time (date and 
time). The data from the keyboard, etc. 508 and the data 
from the internal clock 510 are also supplied to the control 
unit 503. 

[0222] The control unit 503 creates a URL necessary for 
accessing the server based on the supplied ID data as 



described above in the second embodiment, or information 
necessary for retrieval of a bulletin board corresponding to 
a position, registration of a new bulletin board, etc. based on 
the input data, the keyword, the current time data, etc. as 
described above in the third embodiment. Further, the con- 
trol unit 503 creates a display signal for providing a screen 
image on an LCD (liquid crystal display) 507 as a display 
unit, and transmits or receives data to or from the server, etc. 
via a communication unit 509 which serves as a device for 
connection to the modem or the wireless network. 

[0223] With the second and third embodiments of the 
present invention, as described above, since information can 
be easily exchanged using bulletin boards corresponding to 
an object and position in the real world, the user can obtain 
the latest information about a certain object and position, 
and can easily inform other users of new information. Also, 
with the second and third embodiments of the present 
invention, the present system can also be utilized as an 
information system for presenting product support informa- 
tion, advertisement information about a particular place, and 
so on. Moreover, finks (such as advertisements of associated 
products or shops) related to an object or position corre- 
sponding to some bulletin board can be additionally dis- 
played on the bulletin board. Thus, the system of the present 
invention is also valuable as an advertisement medium for 
distributing information to a community of users who are 
interested in a certain object or position. 

[0224] According to the information processing apparatus 
and method, the information processing system, and the 
providing medium of the present invention, as fully 
described above, since attached information is inputted as 
corresponding to specific information specifying a situation 
in the real world, the attached information can be virtually 
attached to the situation in the real world. Also, the attached 
information is information that is related to the situation in 
the real world and can be browsed and exchanged among 
users. According to the present invention, therefore, infor- 
mation (such as image information, text information and 
voice information) corresponding to situations (such as a 
position and an object) in the real world can be presented to 
users. In addition, users can not only attach the information 
to the situations in the real world, but also quickly and easily 
find out desired information. 

[0225] Further, according to the information processing 
apparatus and method and the providing medium of the 
present invention, since attached information is stored in 
correspondence to a situation in the real world, the infor- 
mation corresponding to the situation in the real world can 
be provided to users. According to the present invention, 
therefore, information (such as image information, text 
information and voice information) corresponding to situa- 
tions (such as a position and an object) in the real world can 
be presented to users. In addition, users can not only attach 
the information to the situations in the real world, but also 
quickly and easily find out desired information. 

1. A method for dynamically attaching data items to 
physical environment, the method comprising the steps of: 

capturing surrounding contexts including location level 
context and other level context, wherein the location 
level context and other level context are sensed by 
different types of sensing means; 
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receiving input data item from said surrounding contexts; 
and 

attaching said data item to said surrounding contexts. 

2. A method of claim 1, further comprising step of 
registering said data item as being related to said surround- 
ing contexts. 

3. A method of claim 2 wherein said data item related to 
said surrounding contexts and time information are desig- 
nated to future or past time. 

4. A method of claim 1, wherein said other level context 
is object level context. 

5. A method of claim 1, wherein said capturing step is 
continuously performed so that surrounding contexts are 
always captured. 



6. An apparatus for dynamically attaching data items to 
physical environment, comprising: 

capturing means for capturing surrounding contexts 
including location level context and other level context, 
wherein the location level context and other level 
context are sensed by different types of sensing means; 

input means for inputting digital data item from said 
surrounding contexts; and 

attaching means for attaching said data item to said 
surrounding contexts. 

* * * * * 
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